{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Self-Improving Agent Tutorial\n",
    "\n",
    "## Overview\n",
    "This tutorial demonstrates the implementation of a Self-Improving Agent using LangChain, a framework for developing applications powered by language models. The agent is designed to engage in conversations, learn from its interactions, and continuously improve its performance over time.\n",
    "\n",
    "## Motivation\n",
    "As AI systems become more integrated into our daily lives, there's a growing need for agents that can adapt and improve based on their interactions. This self-improving agent serves as a practical example of how we can create AI systems that don't just rely on their initial training, but continue to evolve and enhance their capabilities through ongoing interactions.\n",
    "\n",
    "## Key Components\n",
    "\n",
    "1. **Language Model**: The core of the agent, responsible for generating responses and processing information.\n",
    "2. **Chat History Management**: Keeps track of conversations for context and learning.\n",
    "3. **Response Generation**: Produces relevant replies to user inputs.\n",
    "4. **Reflection Mechanism**: Analyzes past interactions to identify areas for improvement.\n",
    "5. **Learning System**: Incorporates insights from reflection to enhance future performance.\n",
    "\n",
    "## Method Details\n",
    "\n",
    "### Initialization\n",
    "The agent is initialized with a language model, a conversation store, and a system for managing prompts and chains. This setup allows the agent to maintain context across multiple interactions and sessions.\n",
    "\n",
    "### Response Generation\n",
    "When the agent receives input, it considers the current conversation history and any recent insights gained from learning. This context-aware approach allows for more coherent and improving responses over time.\n",
    "\n",
    "### Reflection Process\n",
    "After a series of interactions, the agent reflects on its performance. It analyzes the conversation history to identify patterns, potential improvements, and areas where it could have provided better responses.\n",
    "\n",
    "### Learning Mechanism\n",
    "Based on the reflections, the agent generates learning points. These are concise summaries of how it can improve, which are then incorporated into its knowledge base and decision-making process for future interactions.\n",
    "\n",
    "### Continuous Improvement Loop\n",
    "The cycle of interaction, reflection, and learning creates a feedback loop that allows the agent to continuously refine its responses and adapt to different conversation styles and topics.\n",
    "\n",
    "## Conclusion\n",
    "This Self-Improving Agent demonstrates a practical implementation of an AI system that can learn and adapt from its interactions. By combining the power of large language models with mechanisms for reflection and learning, we create an agent that not only provides responses but also improves its capabilities over time.\n",
    "\n",
    "This approach opens up exciting possibilities for creating more dynamic and adaptable AI assistants, chatbots, and other conversational AI applications. As we continue to refine these techniques, we move closer to AI systems that can truly learn and grow from their experiences, much like humans do."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports and Setup\n",
    "\n",
    "First, we'll import the necessary libraries and load our environment variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.runnables.history import RunnableWithMessageHistory\n",
    "from langchain.memory import ChatMessageHistory\n",
    "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
    "from dotenv import load_dotenv\n",
    "import os\n",
    "load_dotenv()\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.getenv('OPENAI_API_KEY')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Functions\n",
    "\n",
    "We'll define helper functions for each capability of our agent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chat History Management"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_chat_history(store, session_id: str):\n",
    "    if session_id not in store:\n",
    "        store[session_id] = ChatMessageHistory()\n",
    "    return store[session_id]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Response Generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_response(chain_with_history, human_input: str, session_id: str, insights: str):\n",
    "    response = chain_with_history.invoke(\n",
    "        {\"input\": human_input, \"insights\": insights},\n",
    "        config={\"configurable\": {\"session_id\": session_id}}\n",
    "    )\n",
    "    return response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reflection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reflect(llm, store, session_id: str):\n",
    "    reflection_prompt = ChatPromptTemplate.from_messages([\n",
    "        (\"system\", \"Based on the following conversation history, provide insights on how to improve responses:\"),\n",
    "        MessagesPlaceholder(variable_name=\"history\"),\n",
    "        (\"human\", \"Generate insights for improvement:\")\n",
    "    ])\n",
    "    reflection_chain = reflection_prompt | llm\n",
    "    history = get_chat_history(store, session_id)\n",
    "    reflection_response = reflection_chain.invoke({\"history\": history.messages})\n",
    "    return reflection_response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def learn(llm, store, session_id: str, insights: str):\n",
    "    learning_prompt = ChatPromptTemplate.from_messages([\n",
    "        (\"system\", \"Based on these insights, update the agent's knowledge and behavior:\"),\n",
    "        (\"human\", \"{insights}\"),\n",
    "        (\"human\", \"Summarize the key points to remember:\")\n",
    "    ])\n",
    "    learning_chain = learning_prompt | llm\n",
    "    learned_points = learning_chain.invoke({\"insights\": insights}).content\n",
    "    get_chat_history(store, session_id).add_ai_message(f\"[SYSTEM] Agent learned: {learned_points}\")\n",
    "    return learned_points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Self-Improving Agent Class\n",
    "\n",
    "Now we'll define our `SelfImprovingAgent` class that uses these functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SelfImprovingAgent:\n",
    "    def __init__(self):\n",
    "        self.llm = ChatOpenAI(model=\"gpt-4o-mini\", max_tokens=1000, temperature=0.7)\n",
    "        self.store = {}\n",
    "        self.insights = \"\"\n",
    "        \n",
    "        self.prompt = ChatPromptTemplate.from_messages([\n",
    "            (\"system\", \"You are a self-improving AI assistant. Learn from your interactions and improve your performance over time.\"),\n",
    "            MessagesPlaceholder(variable_name=\"history\"),\n",
    "            (\"human\", \"{input}\"),\n",
    "            (\"system\", \"Recent insights for improvement: {insights}\")\n",
    "        ])\n",
    "        \n",
    "        self.chain = self.prompt | self.llm\n",
    "        self.chain_with_history = RunnableWithMessageHistory(\n",
    "            self.chain,\n",
    "            lambda session_id: get_chat_history(self.store, session_id),\n",
    "            input_messages_key=\"input\",\n",
    "            history_messages_key=\"history\"\n",
    "        )\n",
    "\n",
    "    def respond(self, human_input: str, session_id: str):\n",
    "        return generate_response(self.chain_with_history, human_input, session_id, self.insights)\n",
    "\n",
    "    def reflect(self, session_id: str):\n",
    "        self.insights = reflect(self.llm, self.store, session_id)\n",
    "        return self.insights\n",
    "\n",
    "    def learn(self, session_id: str):\n",
    "        self.reflect(session_id)\n",
    "        return learn(self.llm, self.store, session_id, self.insights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example Usage\n",
    "\n",
    "Let's create an instance of our agent and interact with it to demonstrate its self-improving capabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AI: The capital of France is Paris.\n",
      "AI: Paris has a rich and complex history that spans thousands of years. Its story begins around the 3rd century BC with the Parisii, a Gallic tribe that settled on the banks of the River Seine. The Romans conquered the Parisii in 52 BC, establishing a town known as Lutetia, which flourished as a Roman city.\n",
      "\n",
      "In the Middle Ages, Paris emerged as a center of learning and the arts, home to one of the first universities in Europe, the University of Paris (founded around 1150). The city expanded its influence and infrastructure, including the construction of the iconic Notre-Dame Cathedral, which began in the 12th century.\n",
      "\n",
      "The Renaissance era brought further development, but it was also a period of religious strife, highlighted by the St. Bartholomew's Day Massacre in 1572, where thousands of Huguenots (French Protestants) were killed. The 17th and 18th centuries were marked by the opulence of the Sun King, Louis XIV, and the construction of architectural marvels such as the Palace of Versailles.\n",
      "\n",
      "However, wealth disparities led to unrest, culminating in the French Revolution in 1789. Paris was the revolution's heart, witnessing significant events like the storming of the Bastille and the Reign of Terror. The 19th century saw the city transform under Napoleon Bonaparte and later, under Napoleon III and Baron Haussmann, who redesigned Paris with its wide boulevards, parks, and the iconic landmarks we recognize today.\n",
      "\n",
      "The 20th century was tumultuous, with Paris enduring two world wars, including the Nazi occupation during World War II. The post-war years were a time of reconstruction and cultural renaissance, solidifying Paris's status as a global hub of art, fashion, and gastronomy.\n",
      "\n",
      "Today, Paris is celebrated for its vibrant culture, historical monuments, and significant contributions to art, science, and philosophy. It remains a symbol of beauty, resilience, and the enduring spirit of enlightenment and innovation.\n",
      "\n",
      "Reflecting and learning...\n",
      "Learned: To enhance responses, especially for historical overviews, keep these key points in mind:\n",
      "\n",
      "1. **Prioritize Key Information**: Focus on the most significant events or aspects to keep the narrative engaging and digestible.\n",
      "\n",
      "2. **Incorporate Storytelling Elements**: Use storytelling to create compelling narratives, setting scenes, and highlighting the human aspects of history.\n",
      "\n",
      "3. **Use Clear, Vivid Language**: Employ descriptive language to paint a vivid picture of historical events and transformations.\n",
      "\n",
      "4. **Interactive Elements**: Suggest additional resources like further reading or virtual tours to provide a more in-depth understanding, where applicable.\n",
      "\n",
      "5. **Personalize Responses**: Tailor responses to the user's interests to increase relevance and engagement.\n",
      "\n",
      "6. **Inclusion of Lesser-Known Facts**: Intersperse lesser-known facts or anecdotes to pique interest and provide a unique perspective.\n",
      "\n",
      "7. **Clarity and Structure**: Organize the response clearly, possibly with subheadings or a chronological approach, to enhance understandability.\n",
      "\n",
      "8. **Encourage Interaction**: Conclude with an invitation for further questions, fostering ongoing engagement and tailored information sharing.\n",
      "\n",
      "By applying these principles, responses can be made more engaging, informative, and satisfying for the reader.\n",
      "\n",
      "AI: One of the most famous landmarks in Paris is the Eiffel Tower. Constructed from 1887 to 1889 as the entrance to the 1889 World's Fair, it was initially criticized by some of France's leading artists and intellectuals for its design but has since become a global cultural icon of France and one of the most recognizable structures in the world.\n",
      "\n",
      "Standing at approximately 324 meters (1,063 feet) tall, the Eiffel Tower was the tallest man-made structure in the world until the completion of the Chrysler Building in New York in 1930. It was designed by the French engineer Gustave Eiffel's company. The tower is made of iron and weighs about 10,000 tonnes. Despite its initial intended temporary presence, it was saved from dismantling due to its value as a radiotelegraph station and now attracts millions of visitors each year.\n",
      "\n",
      "The Eiffel Tower has three levels for visitors. Tickets can be purchased to ascend by stairs or elevators to the first and second levels. The journey to the top level offers a breathtaking panoramic view of Paris, making it one of the most visited monuments in the world.\n",
      "\n",
      "Beyond its architectural and engineering significance, the Eiffel Tower has become a symbol of French creativity and ingenuity, representing the spirit of progress and the beauty of Paris to the world. Whether seen up close or from one of the many beautiful vantage points in the city, the Eiffel Tower continues to awe visitors with its imposing structure and the story of its creation.\n",
      "\n",
      "For those interested in exploring more, virtual tours or a visit to the official Eiffel Tower website can offer deeper insights into its history, construction, and the experience of visiting this iconic monument.\n",
      "AI: Another interesting fact about Paris is its nickname, \"The City of Light\" (\"La Ville Lumière\"). This name originated in the 17th century, during the reign of King Louis XIV, when the city began to replace its dark, narrow streets with wide boulevards and installed thousands of gas lamps to illuminate them. This transformation made Paris one of the first major European cities to use street lighting extensively, enhancing its beauty and safety at night.\n",
      "\n",
      "However, the nickname also refers to Paris's leading role during the Age of Enlightenment, a period in the 18th century characterized by intellectual, cultural, and scientific advancements. Paris became a center for education, ideas, and philosophical thought, attracting scholars, artists, and writers from all over Europe. It was during this time that the city truly became a beacon of light, symbolizing hope, progress, and innovation.\n",
      "\n",
      "Today, the moniker \"City of Light\" aptly reflects both the literal illumination of Paris's streets and monuments, as well as its ongoing influence as a center for culture, art, fashion, and gastronomy. The sparkling lights of the Eiffel Tower at night and the illuminated bridges over the Seine River continue to enchant visitors, embodying the city's enduring charm and its historical significance as a beacon of enlightenment and progress.\n"
     ]
    }
   ],
   "source": [
    "agent = SelfImprovingAgent()\n",
    "session_id = \"user_123\"\n",
    "\n",
    "# Interaction 1\n",
    "print(\"AI:\", agent.respond(\"What's the capital of France?\", session_id))\n",
    "\n",
    "# Interaction 2\n",
    "print(\"AI:\", agent.respond(\"Can you tell me more about its history?\", session_id))\n",
    "\n",
    "# Learn and improve\n",
    "print(\"\\nReflecting and learning...\")\n",
    "learned = agent.learn(session_id)\n",
    "print(\"Learned:\", learned)\n",
    "\n",
    "# Interaction 3 (potentially improved based on learning)\n",
    "print(\"\\nAI:\", agent.respond(\"What's a famous landmark in this city?\", session_id))\n",
    "\n",
    "# Interaction 4 (to demonstrate continued improvement)\n",
    "print(\"AI:\", agent.respond(\"What's another interesting fact about this city?\", session_id))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
